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ABSTRACT 

Many schools and school districts are attempting to 
introduce principles and practices of Total Quality Management (TQM). 
These attempts take many forms with varying degrees of commitment of 
resources and management attention. This paper describes the research 
design of a study intended to identify best current practices in 
applying TQM concepts to K~12 school districts across the United 
States. Specifically, the paper describes the sampling procedure and 
data obtained in the first phase of selecting schools and districts 
for later study. A questionnaire sent to a national sample of 205 
schools and school districts involved in TQM elicited A3 completed 
surveys. Three additional respondents sent information (without 
surveys) and five more respondents participated in telephone 
interviews. Criteria for inclusion in the study included: (1) 
involvement of constituents in decision making', (2) a focus on 
instructional processes', and (3) evidence of data collection and 
baseline measures. The questionnaire was not seen as powerful enough 
to determine whether a school was really serious about using TQM to 
make important changes in teaching and learning so a second level of 
screening was pursued. Approximately 22 of the responses met the 
three basic criteria and will be considered for the second level 
qualification stage consisting of follow-up phone interviews. A set 
of schools which will become the sites for longitudinal case studies 
will be selected from this screening. It is argued that TQM has the 
potential to achieve significant change because the organization uses 
the results of its evaluation of outcomes and processes to improve 
practice, and the focus on the client/customer translates into a gain 
in credibility. One figure and the quality improvement practices 
survey are included. The initial analysis of survey data from 
practices in quality improvement in teaching and learning is 
appended. Contains 21 references. (LMI) 
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In current literature, both popular and scholarly, the loss in competitiveness of 
the U.S. in world markets is attributed at least in part, to the inadequacy of our 
educational system. At the same time, there is a great deal of data indicating that 
our public schools are doing more than they did 30 or 40 years ago and are doing it 
better and for more people (Bracey, 1993). Surveys indicate that -me st j direct 
clients or customers of local educational organizations are satisfied with their 
schools while at the same time feeling that the nation's schools on average are NOT 
doing so well (Elam, Rose & Gallop). 

Yet, taxpayers (in a broader sense the customers of public education) are not 
nearly as satisfied as the direct clients of public schools as indicated by the fact that 
they continually vote less and less money to support education (Elam et al 1993). 
Employers, who say they must spend a great deal of money retraining newly-hired 
people, claim that they are not finding employees with the kinds of skills needed to 
function in today's increasingly complex and technological work world (Berryman 
1990, Horan 1993, and Janney-Pace 1993). 

Public schools in the U.S. today are arguably more efficient and perhaps even 
more effective than they were in the pre-Sputnik era. Yet they are not considered 
good enough, efficient or effective enough for today's and tomorrow's needs. Our 
economic productivity growth is not keeping pace with our world competitors, 
especially in Europe and Japan. But how do schools determine what is good enough 
and how to get there? 



This researsh was funded in part by a grant from the Quality Leadership Center 
at the Carlson School of Management, University of Minnesota. 
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Leaders and managers of educational organizations may be somewhat confused by 
the mixed messages they are getting. Who are their clients or customers? -- 
Students? Parents? Local taxpayers? Society in general? Employers of their 
graduates? Colleges and Vocational schools attended by their graduates? Until they 
clearly define who the customer is, how can they define and improve quality? Can 
they continue to serve multiple constituents within the present system? School 
improvement becomes an elusive goal, depending on who is defining 
improvement and their method. 

In recent years, many techniques and popular concepts have been introduced to 
school systems. They often have three-letter acronyms (e.g. SBM— site based 
management; OBE— outcome-based education) and may be based on good theory but 
have little practical application assistance and almost no tie-in to performance (e.g. 
empowerment, shared decision making, computer literacy, multi-disciplinary team 
teaching, restructuring, etc. rather than measures of student achievement and 
performance). One would expect that the rationale for these interventions, though 
it is usually not stated, would be the possibility of improving some kind of 
performance outcomes for schooling. But little true school change (i.e. changes in 
the delivery of teaching and learning at the classroom or performance level) seems 
to have resulted from the application of the techniques and concepts noted above, 
though the nation's schools have spent hundreds of millions of dollars on 
consultants and training seminars to implement such "programs." For a summary 
of the research on school based management's (lack of) impact on student 
achievement see Peterson (1991). 

In short, there has been a great deal of frustration over the failure of school 
reform, efforts that began with the "Nation at Risk" report (1983). Scholars such as 
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Sarason (1990) provide many theories for why reform efforts are failing. According 
to a prominent management scientist, Russell Ackoff, "The problem with education 
is that society is trying to improve the wrong system rather than making the system 
right" (1993). 

Further complicating the problem is the fact that very little research in 
education administration has focused specifically on outputs, as of yet. Some very 
general research has been conducted on the impact of processes and behaviors on 
output (effective schools literature), but this has been hard to apply. This is true 
because of the difficulty in measuring the processes (e.g. high expectations, safe and 
orderly climate, collegiality, strong instructional leadership, parental involvement, 
well-defined goals, and a system for monitoring progress, etc.) and the narrowness 
of the output measures (typically standardized reading and math test scores and 
often only in urban schools) that have often been used as the dependent variables. 

A recent summary of the literature linking processes to outcomes is a "meta- 
review" and synthesis of the research on variables related to learning done by 
Wang, Haertel and Walberg (1990). This study provides some excellent guidance to 
those who might be trying to use any technique for improving student output. 
They found that the variables most strongly related to learning were student factors 
(metacognition and peer group influence), classroom instruction variables 
(classroom management, quantity of instruction, student-teacher social interaction), 
and classroom climate. They netted the minimal effect of state and district policy 
variables and adminisr-ative decision-making on student learning. 

Aside from this "meta-review" which included some empirical study, there 
has been little empirical data from schools on the relationship of processes to 
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outcomes. A recent study found that in research on school outcomes, both in North 
America and Europe, most theoretical models used to interpret findings are 
unprepared Lo address conversion of process variables to outcomes. Most are built 
on the assumption that outcomes can be attributed to something in the system or 
institution, and the absence of linkages back to the learning environment leaves 
internal decision makers without information to correct causes that explain 
variations in quality (Nedwick and Neal, 1993). 

One approach that is believed to have the potential for helping to address some 
of the shortcomings in the reform efforts to date is the application of "Quality.", 
concepts in school systems. Because "Quality" concepts force an organization to 
think in terms of processes and outputs rather than inputs, they can be especially 
useful to educational organizations, which traditionally have emphasized inputs. 
For example, credit hours, seat time, credentials of teachers, pupil, teacher ratios, and 
legislated number of class days have been important indicators in the highly 
regulated monopolistic context of public education. 

One recent article on the application of TQM points out a caution in trying to 
apply the industrial quality "control" model to an educational organization. It 
points to the need for a different approach (the authors call it "Total Quality 
Learning" or TQL) when applying "Quality" concepts to conditions involving high 
task uncertainty. Despite this caveat, application of the "Quality" process, or TQM, 
unlike most of the school reform techniques and attempts at systems change 
introduced thus far, seems to have a real chance for achieving some important 
changes in the processes and outcomes of schooling for at least two key reasons. 
First, the application of "Quality" concepts forces an organization to begin 
measuring outcomes AND processes - and TO USE THE RESULTS OF THESE 
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MEASURES TO IMPROVE PRACTICE. With results measures and process 
measures in hand, continuous improvement goals can be set and the natural 
internal motivation of most educators can be activated to work more effectively 
toward these goals. Second, by focusing some attention on the client or customer, 
the schools could begin to gain much more credibility and in turn probably be 
rewarded with more resources to perform their socially vital jobs more effectively. 
Properly applied, it is said that "Quality" concepts will lead to the kinds of systems 
changes that schools must undergo if true reform is to take place. 

To have the desired effect on student and school performance, any reform or 
restructuring effort must address important process variables and try to ascertain 
their relation to outcomes and then work on enhancing the impact of these factors 
on student achievement. The same, of course, is true of TQM. 

Therefore, empirical studies of TQM efforts are needed to see if data that truly 
relate to meaningful teaching and learning processes and outcomes are really being 
collected. If so, are these data being used to improve teaching and learning. Until 
recently, the research and writing on "Quality" in schools has been chiefly 
normative or prescriptive in nature, drawing mostly on business for its examples 
and empirical data. There is a need for empirical work in schools and other 
education organizations to identify the impact over time of the application of 
quality principles in these kindt of organizations. (Murgatroyd & Morgan, 1993, 
Lezotte, 1993). Since true reform and system restructuring could take five to ten 
years before visible results can be observed, in most cases, it may be too early in the 
implementation stages to learn a great deal about the effectiveness of the 
applications. Our hope, however, is to begin a process of longitudinal tracking of 
implementation efforts over time. To do this, we have to try to identify those 
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schools or school districts that seem to have the most promise of producing useful 
results. 

Building on the idea of total quality learning and also cognizant of the fact that 
any attempts at school reform must have some direct impact on ine ud. urn and 
on teaching and learning, we launched a research program with the long run 
objective of looking at how "Quality" concepts might be affecting or might have the 
potential to affect teaching and learning, and ultimately affecting system change. 
We plan to identify best current practices in applying "Quality" concepts available in 
K-12 school districts in this country. We also propose to study the evolution over 
time of these practices and the practices in school districts not yet as far along, with a 
view toward gaining better understanding of the opportunities, pitfalls, and 
potential benefits of various approaches used in implementation. 

The first phase in our program was to identify the schools in the country who 
seem to be applying in a serious and thoughtful way, the principles of total quality 
management. At this stage we were not trying to look at what approaches they used 
(Baldrige, Deming, Juran, 3-M, or others), but rather whether they were applying 
some key general concepts such as the tracking of processes that might be related to 
teaching and learning, a focus on client needs, monitoring of processes and 
outcomes and then using data for continuous improvement efforts. 

Many schools and school districts are currently attempting to introduce 
principles and practices of total quality management. These attempts take many 
forms with varying degrees of commitment of resources and management 
attention. This paper reports on how we identified our sample and reports on the 
data we obtained in our first phase in which we are "qualifying" schools and districts 
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for later study. 



The next phase of our work will be to do detailed case studies in a very small 
number (4-6) of schools which are at various stages of development of the 
application of "Quality" concepts. In these studies we will attempt to see the 
development of these efforts over time and try to identify the factors that lead to 
changes in teaching and learning and in systems change. We also want to find out if 
schools try to relate any processes to outcomes and of course what outcomes they 
track over time - such as test scores, drop-out rates, attendance, participation rates, 
college admissions. Even more importantly we want to find out how they 
determine which outcomes to track to see if they are truly trying to become more 
sensitive to customer or client needs. 



METHODOLOGY 



We first selected a national sample of schools to which we sent a preliminary 
questionnaire. The sample was selected by using lists frcn the Air rican 
Association of School Administrators (quality list), the American Society for Quality 
Control, the Minnesota Academic Excellence Foundation (Partners for Quality), the 
3rd Annual National Governors' Quality Conference and added schools identified 
in the literature as being involved in TQM. A total of 205 schools and districts were 
identified. 

The following mailings were sent to contact persons at each of the 
schools /districts in the database (205): 

1. a pre-survey postcard - which explained the purpose of the research, 
the people involved, and that the respondent was chosen to 
participate and would be receiving a survey. 
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2. survey packet - included the "Practices hi Quality Improvement in 
Teaching and Learning" survey (see exhibit vine), a reply postcard (see 
exhibit two), a cover letter, and a stamped return envelope. 

3. thank you letters will be sent to those who returned the completed 
survey, and we will be contacting many of them in the future as we 
gather longitudinal data. 



The questionnaire (exhibit one) was designed to obtain brief general 
answers to the questions dictated by our framework noted above and 
attempted to answer the following: 

• Processes tracked (researchers judgment on their relation to 
teaching and learning) 

• What outcomes are tracked and who decided on their importance 
(Are external clients considered?) 

• What is done with the data collected? Is it reported back for action? 

• How does their resource allocation to staff development reflect their 
commitment to effectively apply "Quality" concepts? 

• Is input from external clients used to influence the measurements 
employed and the subsequent action steps? 



RESPONSES 



We received 77 reply postcards back. Several of the respondents (4) were 
forwarding the survey t(\sorr?one else in their school, two returned saying 
that they were not a school per se, 15 said they would not reply (stating 
various reasons including not enough time and we do not track or do 
continuous improvement), three respondents said that they would return the 
survey later, and eight said they would be willing to respond by engaging in a 
phene interview. 
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After a two-week period, follow-up phone calls were attempted to all 
those who had not responded in some way (to remind people to return the 
survey). Approximately 60 people were actually reached. 

From the 205 who were sent a survey, we received 43 completed surveys. 
We received packets of information from three respondents (without 
surveys), and we were able to complete 5 phone interviews (with those that 
said they were willing to reply by phone rather than fill out the survey). A 
total of 51 respondents provided data for the research and for selection of the • • 
schools for future detailed field work. 



• Number mailed: 205 

• Completed questionnaires returned: 43 

• Additional materials returned: 3 

• Phone interviews: 5 

• T otal Responses : 51 



The phone interviews (5) were conducted following the questions from the 
original questionnaire. 

Initial Screening 

Since the chief purpose of this initial survey was to identify potential schools 
for our later research, we needed to apply specific criteria to screen the surveys. In 
trying to qualify schools for our later case study we thought several criteria were 
important. It was not vital to us that they identified themselves as "doing TQM", 
but was more important that they gave us some evidence that they were really 
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tracking progress or changes in process variables that might have the possibility of 
truly influencing teaching and learning. 

The responses were screened on several criteria based on research on the 
implementation of TQM in education. The basic criteria were: 1) 
involvement of constituents (parents, community members, students, 
teachers, and administrators) in decision-making, goal-setting, or general 
input; 2) a focus on instructional processes (as opposed to only looking at 
outcomes); and 3) evidence of data collection and baseline measures 
(benchmarking, tracking processes, etc.). In analyzing the responses, some 
evidence that each of these three conditions existed was necessary to meet the 
initial screening qualifications. 

We decided our questionnaire v/as not powerful enough to determine 
whether a school was really serious about using TQM to make important 
changes in teaching and learning. So we decided to pursue a second level 
screening. Approximately 22 of the responses met these three basic criteria 
and will be considered in the second level qualification stage. This will 
consist of follow-up phone interviews designed to better understand the 
responses given by several of the 22 schools who are in the first cut, where we 
felt unsure of our decision. 

After the secondary screening, we will select a set of schools which will become 
the sites for our longitudinal case studies. These are the schools in which we will 
perform detailed field work. We will select one or two schools which have been 
involved the longest and which seem by initial screening, one or two schools which 
have just begun and one or two schools which are in their second, third or fourth 
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year of design and implem. .Jon work. The case study approach as outlined by 
Yin (1994) will be the dominant methodology for this part of our study. 

The framework for these studies will include the variables outlined by Wang 
et. al. in their meta-research on student learning. We will try to see it b .ent, 
classroom, community context, and teacher-behavior variables are being studied, 
tracked, reported, and whether the results are being acted upor ; n a meaningful 
way. 



DATA ANALYSIS 



Since this is not a survey administered for the purpose of gathering statistical 
data, we only report the responses for information where they seemed to have 
special interesc or significance. The main use of the data was to help us select the 
sample for the next step and to help design the protocols for the field work to be 
done. See appendix A for a summary of the findings. We are now in the process of 
designing the field work procedure to be used in collecting the case studies. We 
hope to begin these case studies next year. 



FRAMEWORK FOR CASE STUDIES 



Based on a study of the research and data collected in our survey, we will 
develop a framework and protocol for the field work. Our initial thoughts on what 
that framework may look like is depicted as follows: 
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businesses 



staff 



parents 



students 




administrators 

other 



Outcome Goals 



INPUTS 

•student 
characteristics 




customers 



PROCESSES 

• student behavior variables 
1 classroom instruction 
variables 

1 policy variables 

• parent behavior 



OUTCOMES 

•learning 
•achiev ».ment 
•growth 



PRODUCT 



The output of the case study research should include a detailed examination of 
the kinds of implementation programs that seem to be headed toward significantly 
changing what happens in the classroom in teaching and learning. We should also 
be able to include a set of recommendations for administrators on (1) how to begin 
the quality journey, pitfalls to avoid, best approaches for success, obstacles, reasons 
for potential failure; (2) approaches used to keep motivation and interest high 
during implementation; and (3) how schools have institutionalized the concepts. 
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According to Wolf and Leader, "in school districts where total quality has beew 
successfully implemented, it fosters shifts in thinking about the structure of 
educational systems and where they fit in the total community. People see their 
roles differently; business, education and the community recognize their 
interdependence; and schools no longer deliver only the basic skills. They strive to 
deliver education that provides skills more in line with those identified in the 
SCANS report" ( (1993, p. 19). 

Therefore, another way in which we will attempt to identify effective use of 
TQM is by noting whether the schools are incorporating local community and 
business views, and whether or not the school is discussing and tracking processes 
that signify an awareness of the importance of the interdependencies among school, 
community, business, and teaching and learning. 

We will be especially concerned with whether there is some attempt to look at 
the processes which affect teaching and learning and to identify what these might be. 
For example, do the school's constituents reviewing literature such as Wang et al 
(1990), or gather and analyze some kind of local data on student characteristics, 
school context, classroom methods, and home and community influences? Or do 
they gather and analyze data on other variables that researchers or practitioners 
judge as being important to student performance and achievement, or as being 
important to achieving other outcomes that constituents wish from the school? 

It may take several years to answer our basic question, "Does TQM affect 
teaching and learning?", but our research program will have as its ultimate goal to 
find out if and how this happens. 
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Exhibit #1 



PRACTICES 
IN 

QUALITY 
IMPROVEMENT 

IN 

TEACHING 

AND 
LEARNING 



A research project of the 
the Bush Educators' Program, 
the Quality Leadership Center 
(of the Carlson School of Management, University of Minnesota) 
and cooperating K-12 schools across the United States 



PLEASE RETURN TO: 
Bush Educators' Program 
Carlson School of Management 
University of Minnesota 
271 19th Avenue South 
Minnneapolis, MN 55455 
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QUALITY IMPROVEMENT PRACTICES SURVEY 

Brief answers, lists, outlines, or attachments would be sufficient responses to the questions. 
Please feel free to attach forms or documents used in your schools in answering the 
questions below. 

YOUR NAME AND TITLE . 

SCHOOL AND DISTRICT NAME . 

ADDRESS . 

NUMBER OF STUDENTS IN YOUR SCHOOL DISTRICT 

AS YOU ANSWER THESE QUESTIONS ARE YOU THINKING OF YOUR: 

Classroom(s) Yourself (title: ) 

School Building School District 



1 . List a few of the key outcomes on which you gather data. 



2. Have external constituents had input into the selection of outcomes to be tracked? If so, how? 
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3. What has been the primary locus (if any) of your staff development activities over the last two years'' 



4. How are staff development needs determined? 



5. Do you gather data on constituent perceptions? If so, how often and what do you do with the data? 
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6. What data on teaching and learning processes do- yon track over time? (e.g. teaching methods, class 
size, quantity of homework assigned, parental involvement, other) 



7. How do you use data you have on processes and outcomes? 



8. Describe briefly the use of any Total Quality Management tools or processes in your 
school (district) not already mentioned above. 



OPTIONAL- ATTACH COPY OF ANY FORMS OR GRAPHS YOU USE THAT WOULD HELP 
EXPLAIN ANSWERS TO ANY OF THE ABOVE QUESTIONS 
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REPLY CARD 

. I AM RETURNING THE COMPLETED SURVEY 

WILL REPLY LATER BUT DO NOT HAVE TIME NOW 

WILL NOT REPLY. SORRY, TOO MANY QUESTIONNAIRES 

WILL TAKE A CALL AND PROVIDE BRIEF ANSWERS BY PHONE 

WILL FORWARD THE SURVEY TO (name/title) 

TO FILL OUT FOR ME 

Your Name 

School 

Address 

Phone # • 
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APPENDIX A 



INITIAL ANALYSIS OF SURVEY DATA FROM 
PRACTICES IN QUALITY IMPROVEMENT IN TEACHING AND LEARNING 

The Practices In Quality Improvement In Teaching and Learning Surv.-.y 
sought general, informational data on a number of items directly related to quai, ;y 
improvement practices in schools identified as leaders in total quality educational 
programs. Attempting to initiate a set of non-leading, open-ended questions, while 
trying to determine what outcomes were actually being tracked, the survey began 
with the request 'List a few key outcomes on which you gather data.' 

Tabulated results indicated that nearly 75% of the respondents listed academic 
or learning related items as being key outcomes on which they gathered data. The 
terms used by the respondents to describe these academically related outcomes 
included test scores, attendance, drop-out rates, course outcomes, instructional 
methods, assigned homework, graduation rate, and student masteries or 
competencies. One quarter of the respondents named specific curricular areas on 
which they gathered data. 

Approximately one-third mentioned organizational or system related 
outcomes as being key indicators they gathered data on. These areas were identified 
by terms such as staff development, goal setting, budget, climate, and department 
specific areas such as custodial, food service or transportation. 

Nearly one-half of the respondents listed factors relating to non-academic, but 
still school related outcomes as being factors on which they gathered data. These 
areas included client surveys, customer satisfaction, parent involvement, discipline, 
job placement, college admissions and graduation, and specific student attributes 
including self-esteem, motivation, mobility, participation and responsibility. 
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The survey asked for specifics regarding the tracking of data on teaching and 
learning, and offered examples such as teaching methods, class size, quantity of 
home assigned and parental involvement. Even with the request for and examples 
of specifics, a number of respondents gave rather vague answers. Several stated that 
they do not track any specific processes. 

Of those who do regularly track data, no single process appeared universally 
popular. Class size was the most often mentioned area tracked, with thirty-nine 
percent of the respondents indicating that they kept data on class size. Teaching 
methods and test scores were close followers with thirty-four and thirty-two percent 
respectively, and parental involvement was tracked by twenty-seven percent of the 
respondents. 

The responses indicated a much higher level of agreement on what was done 
with the data once it was collected. Here, over forty percent of the respondents 
included the words "make improvements" in describing their use of the data they 
had collected. Additionally, nearly twenty-five percent stated that they used the data 
to make "decisions." One must assume that these decisions also represented 
improvements to the educational systems, bringing the combined total using data 
for decision making improvements to sixty-eight percent. 

A considerably smaller number, seventeen percent, stated that they used the 
data collected to "share" or "inform". Finally, twelve percent said that they used the 
data to "measure" but gave no other indication what they did beyond measuring. 

Staff development was identified by the research team as being a key indicator 
of quality improvement practices both from the standpoint of its close association 
with the teachers and the craft of teaching, and because of the ability and 
responsibility of many staff development teams to conceptualize and commit to 
practice essential principles of system changes. The questionnaire first asked what 
the primary focus of the staff development activities had been over the past two 
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years, and then asked how staff development needs were determined. 

In describing the focus of their staff development team, sixty-six percent of the 
respondents included the terms "quality" or "improvement" in the team's 
responsibility. Several stated that the staff development team's primary purpose 
was to "facilitate leadership" and "deploy" Total Quality Management techniques in 
the school. 

Yet when asked how staff development needs were determined, little 
uniformity could be found. Some mentioned staff surveys, while others noted state 
mandates. A few relied on building-site interests to determine staff development 
needs, and others used district-level steering committees. A small number said they 
were dependent on "district goals", while others focused on "visionary leadership" 
and "emerging trends." Finally, when answering how staff development needs 
were determined, one respondent said "Not very well!"; another simply said "We 
sit down together and talk." 

The final set of data gathered by the questionnaire dealt with the impact of 
external constituents. Again, two questions were asked. One asked whether data 
was gathered on constituent perceptions and the other on whether constituents had 
input into the selection of outcomes to be tracked. Both questions included follow- 
up requests for how this data, if gathered, was used. 

The idea of gathering constituent data appeared to be a popular activity with 
Total Quality schools. Eighty-two percent of the respondents indicated that they 
regularly gathered constituent data. The constituent groups named by the 
respondents varied considerably, and included staff, students, parents, 
administrators and community groups, with parents and being the most often 
mentioned group. Frequency of data gathering also had a wide variance, ranging 
from once every three years to as often as four times per year. 

Respondents were less clear about what they did with the data once they had 
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collected it. Several stated that they used the data to help determine a focus for 
improvement and some used it for setting district goals. Others said they used the 
data for modifying programs, a couple mentioned it was used to focus issues for staff 
development, two said it was given to the teachers, and another stated that it was 
given to the school board. 

Over three-quarters of the schools responding indicated that they had external 
constituents giving input into the selection of the outcomes that were to be tracked. 
Again, there was little common ground shared among the respondents as to what 
purpose this input by outside constituents had. Several noted that it helped in 
planning, and three specifically mentioned strategic planning. Goal setting wa: 
listed by two respondents, and selecting standards or outcomes was named by six 
respondents. Other uses of constituent input included targeting areas for 
improvement, creating school report cards, and determining what students should 
be able to do when they graduate from high school. 
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